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PREFACE 

This  Memorandum  is  part  of  RAND's  continuing  program  to  develop 
basic  analytical  techniques  for  application  to  Air  For:'  problems. 

The  validity  of  many  economic  relations  derived  by  applying  the 
ordinary  linear  least-squares  regression  method  to  time  series  is  of¬ 
ten  questionable,  because  the  implicit  assumption  of  serially  indepen¬ 
dent  disturbances  cannot  be  justified  The  principal  alternative  mod¬ 
el  considered  assumes  that  the  disturbances  are  generated  by  a  first- 
order  autoregressive  process.  Several  estimators  under  the  latter  as¬ 
sumption  have  been  suggested,  but  little  is  known  about  their  small 
sample  properties.  This  study  describes  the  relative  performance  of 
the  estimators  based  on  results  of  a  Monte  Carlo  experiment. 

The  Memorandum  is  intended  for  operational  and  economic  analysts 
who  deal  with  time  series  data.  It  is  assumed  that  the  reader  is  fa¬ 
miliar  with  basic  econometric  literature  on  time  series  analysis  Two 
potential  areas  of  Air  Force  application  are  manpower  prediction  and 
demand  prediction  for  spares. 

Clifford  Hildreth  is  a  consultant  to  the  Logistics  Department  of 

RAND. 


-v- 


SUMMARY 


Economists  interested  in  analyzing  time  series  have  long  recog¬ 
nized  autocorrelated  disturbances  ag  one  of  the  principal  hazards  that 
may  cause  serious  inefficiencies  In  their  analyses.  In  the  past  de¬ 
cade,  several  econometricians  have  studied  a  statistical  model  in  which 
the  disturbances  are  assumed  to  be  generated  by  a  simple,  first-order 
autoregressive  process,  and  have  proposed  several  estimators  of  the 
unknown  parameters.  Because  little  is  known  of  the  probability  laws 
governing  the  estimators  and  therefore  of  their  relative  desirability 
in  various  circumstances,  and  because  determining  the  laws  analytically 
poses  severe  problems,  a  study  of  the  behavior  of  alternative  estima¬ 
tors  applied  to  artificially  generated  data  with  known  parameter  values 
was  undertaken,  and  its  results  are  reported  herein. 

To  generate  artificial  data  for  this  experiment,  eight  structures 
were  specified.  Each  structure  differs  from  the  others  in  one  or  more 
of  the  following  aspects:  the  pattern  of  observed  values  of  the  inde¬ 
pendent  variables  (these  are  arranged  in  a  matrix  denoted  by  Z) ;  the 
value  of  the  autocorrelation  coefficient  (e);  and  the  sample  size. 
Samples  of  size  30  were  drawn  for  fou.  structures  and  samples  of  size 
100  were  drawn  for  four  others.  For  each  structure,  300  samples  of 
the  selected  size  were  drawn  and  estimates  of  unknown  parameters  were 
calculated  for  each  sample  by  five  different  methods.  They  were  Maxi¬ 
mum  likelihood  (MI.)  estimators,  Theil-Nagar  (TN)  estimators,  approxi¬ 
mate  Bayes  ( AB )  estimators,  Durbin  (D)  estimators  and  Least  squares 


(LS)  estimators. 
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AnaJyses  of  the  performance  of  the  above  five  estimators  leads 
to  several  general  observations: 

1.  When  p  is  nonnegative,  ML,  TN  and  D  estimators  all  have  a 
persistent  tendency  to  underestimate  p  on  the  average. 

2.  Judging  by  the  absolute  deviations  of  sample  means  from  their 
respective  true  values,  the  TN  estimator  of  p  Looks  slightly 
better  for  samples  of  size  30  and  relatively  small  p,  i.e., 

|p|  1  0.3;  however,  Ml.  appears  to  perform  better  for  samples 
of  size  100  and  relatively  large  absolute  values  of  p.  The 
Durbin  procedure  appears  a  little  less  biased  than  TN  for 
samples  with  100  observations,  but,  in  general,  it  appears 
least  favorable  among  the  three  estimators. 

3.  The  TN  estimator  of  p  has  a  smaller  variance  than  the  ML  es¬ 
timator  for  samples  with  30  observations  and  relatively  small 
p.  However,  the  variance  of  the  ML  estimator  is  smaller  for 
sampi',t  of  100  ofcpervofionq  and  relatively  large  p. 

.  On  the  average,  the  D  estimator  of  p  has  a  larger  variance 
than  the  other  two  estimators. 

5.  The  sample  means  of  all  the  estimators  of  y's  are  similar  and 
are  close  to  their  true  values,  even  for  samples  with  as  few 
as  30  observations. 

6.  Judging  by  low  mean  square  error,  TN  estimates  of  y's  are  a 
little  better  than  ML  for  samples  with  30  obssrvstions ,  but 
for  samples  with  100  observations,  both  estimators  perform 
about  the  same,  Ths  D  estimator  is  slightly  worsa  than  both 
ML  and  TN  estimators  regardlsss  of  sample  size. 

7.  For  samples  with  only  30  observations  and  p  as  large  as  0.3, 
the  other  three  eetimatore  do  not  have  advantages  over  the 
LS  estimator.  LS  also  estimates  coefficisnts  well  when  the 
columns  of  Z  ere  "smooth." 

Besides  examining  ths  performance  of  various  astiraators,  we  also 
checked  the  behevlor  of  several  commonly  used  tests  of  lndependencs 
of  regression  disturbances.  Ths  tests  conaidared  were  the  Von  Neumann 
ratio  teat,  the  Durbin-Wetson  teet,  the  Thail-Nagar  test,  the  likeli¬ 
hood  ratio  tael,  end  the  teet  baaed  on  the  asy«pt>.  j.c  distribution  of 
the  ML  estimate  of  o  (we  shell  call  this  the  £  teat).  Some  general 
observations  ere  as  follows: 

6.  There  were  many  Inconclusive  applications  of  the  DW  test,  as 
previously  noted  by  both  theorists  and  practical  workers. 
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9.  For  the  sample  sizes  used  in  this  study,  the  TN  test  amounts 
to  rejecting  the  null  hypothesis  in  those  cases  where  the  DW 
test  either  rejects  or  is  inconclusive.  Inspection  of  the  TN 
and  DW  tables  reveals  that  this  will  be  virtually  true  except 
for  quite  small  samples. 

10.  TN  rejected  a  true  null  hypothesis  much  too  frequently  for 
samples  of  size  30. 

11.  The  tendency  noted  above  for  ML  to  underestimate  p  was 
reflected  in  low  frequencies  of  rejection  of  true  null  hypo¬ 
theses  by  one-tailed  (5  testa  and  high  frequencies  for  two- 
tailed  tests.  Thus,  the  (5  test  cannot  be  recommended  when 
based  on  the  asymptotic  distribution.  In  considering  this 
bias  in  the  actual  significance  level,  however,  the  rejec¬ 
tion  rates  for  false  hypotheses  were  relatively  large.  This 
suggests  that  a  powerful  test  can  be  based  on  0  if  a  good  ap¬ 
proximation  to  its  finite  sample  distribution  can  be  found. 

Hildreth  [13]  has  shown  that  the  ML  estimators  are  asymptotically 
normal  and  that  the  vector,  ?,  of  estimates  of  coefficients  is  asymp¬ 
totically  independent  of  £,  0,  the  estimators  of  the  autocorrelation 
coefficient  and  the  variance.  It  was  conjectured  that,  for  many  pur¬ 
poses,  the  asymptotic  distribution  of  y  would  prove  a  tolerable  approx¬ 
imation  in  the  sample  sizes  often  encountered  in  econometric  studies, 

but  that  for  (5  and  0  the  asymptotic  distributions  would  be  le98  satis- 

2 

factory.  This  tends  to  be  confirmed  by  the  x  goodness-of-f it  statis¬ 
tics  computed  from  the  generated  data. 

In  conclusion,  the  reader  must  b«  aware  that  the  above  ooaeiva- 
tions  are  descriptive  statements  of  how  certain  statistics  behaved  in 
this  particular  experiment.  Since  300  samples  were  drawn  for  each 
structure,  we  hope  that  the  observed  characteristics  are  generally  rep¬ 
resentative  of  these  structures.  The  characteristics  of  the  \ccious 
structures  were  chosen  to  represent  a  variety  of  circumstances  that 
might  reasonably  be  encountered  in  practical  work.  To  know  just  how 
representative  the  structures  are,  however,  would  require  a  careful 
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survey  of  applications,  and  this  has  not  Man  undertaken.  It  is  de¬ 
sirable  that  hints  furnished  by  a  study  such  as  this  be  supplanted  by 
analytical  results  whenever  poss  e.  For  inroortant  prooerties  that 
remain  intractable  after  further  theoretical  analysis,  additional  Monte 
Carlo  experiments  are  in  order. 
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I.  INTRODUCTION 

Economists  have  long  been  concerned  that  nonindependent  distur¬ 
bances  may  be  a  frequent  cause  of  inefficiency  in  estimates  of  regres- 

4- 

sion  coefficients  for  time  series.  In  the  past  decade,  several  econ¬ 
ometricians  have  studied  an  alternative  model  in  which  the  disturbances 
are  assumed  to  be  generated  by  a  simple,  first-order  autoregressive 
proces. ,  and  have  proposed  several  estimators  of  the  unknown  oarameters. 

Because  little  is  known  of  the  probability  laws  cf  the  estimators 
and  therefore  of  their  relative  desirability  in  various  circumstances, 
and  because  determining  the  laws  analytically  poses  severe  problems, 
we  undertook  a  study  of  the  behavior  of  alternative  estimators  applied 
to  artificially  generated  data  with  known  parameter  values.  Such  studies, 
of  course,  furnish  hints  rather  than  conclusions  about  the  behavior  of 
various  statistics.  The  investigator  determines  certain  structures  in 
advance  and  generates  samples  by  drawing  random  components  according 
to  a  specified  probability  law,  with  the  aid  of  tables  of  random  num¬ 
bers  or  other  random  devices.  The  results  may  be  misleading  because 
of  special  features  of  the  structures  chosen  or  bacauae  statistical 
accidents  occur  in  generating  samples  (18,  especially  pp.  3-5]. 

The  hints  from  a  particular  study  can  be  strengthened  by  drawing 
many  samples  for  each  structure  (thus  insuring  a  low  probability  of 
misleading  statistical  accidents),  and  by  examining  a  wide  array  of 
representative  structures.  Of  course,  each  tactic  incrsases  ths  re¬ 
sources  needed,  and  the  study's  final  design  is  always  a  compromise 


See  [3,  6,  7,  17,  21]. 
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between  the  cost  of  resources  and  the  desire  to  make  the  results  as 
reliable  as  possible. 

In  this  study,  eight  structures  were  chosen,  with  samples  of  size 
30  drawn  for  four  structures  and  of  size  100  for  four  others.  For 
»ach  structure,  300  samples  of  the  selected  size  were  drawn,  estimates 
of  unknown  parameters  were  calculated  for  each  sample,  by  alternative 
methods,  and  characteristics  of  the  resulting  frequency  distributions 
of  estimates  were  calculated  and  tabulated. 

Section  II  completes  a  sketch  of  the  study's  design  end  gives  rea¬ 
sons  for  some  of  the  choices.  Section  III  presents  and  discusses  the 
study's  results.  Appendices  A  through  D  describe  in  some  detail  the 
methods  used  to  generate  artificial  data  and  to  obtain  the  maximum 


likelihood  estimator. 
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II.  DESIGN  OF  THE  STUDY 


The  model  employed  specifies  that  an  observed  vector  y  of  order 

equal  to  the  sample  size  (30  or  100)  comes  from  a  multivariate  normal 

population  with  mean  vector  Zy  and  variance  matrix  vA, 

where  Z  :  a  known  matrix  of  order  T  x  K  representing  T  observed  values 
of  each  of  the  K  independent  variables; 

y  :  a  vector  of  K  unknown  coefficients  to  be  estimated; 

A  :  a  T  '■  T  matrix  with  typical  element  a  -  [  1/ (1-p^)  ]p  ^  C  8^; 

S  w 

p  ;  a  constant,  |p|  <1,  called  the  autocorrelation  coefficient; 
\>  -  a  positive  constant. 

The  interpretation  is  that  an  element  y  of  y  is  determined  as  a 
linear  combination  of  corresponding  elements  of  2  plus  a  disturbance 
that  is  lineally  relate'  to  the  disturbance  of  the  preceding  observa¬ 
tion;  i.e.  , 


(1) 


K 

yt  ■  l  \kTk +  ut  •  where 

k-1 


(2) 


ut  -  dut_^  +  vt  ,  t  •  2,  3,  ...»  T, 


ui  ’  7==^t  ’ 

A  -  o 

the  v  are  normal,  identical,  and  independent  with  mean  0  and  variance 
v . 

We  chose  a  sample  size  of  30  for  four  structure*  because  many 
studies  of  economic  time  series  involve  20  to  40  observations.  We 


chose  100  as  the  other  sample  size  because  autocorrelation  is  very 
likely  to  be  present  In  quarterly  or  monthly  data,  and  in  these  cases 


the  sampis  size  may  be  much  larger — 100  or  more  is  not  uncommon.  And 
it  seemed  desirable  to  have  two  sample  sizes  far  enough  apart  so  that 
we  might  note  any  tendencies  for  asymptotic  properties  to  be  more 
nearly  rea' Lzed  in  the  larger  samples. 

Past  theoretical  studies  [2,  A,  6]  show  that  properties  ol  some 
suggested  procedures  depend  critically  on  the  value  of  p  and  on  the 
pattern  of  Z.  It  therefore  seemed  useful  to  arrange  a  set  of  struc¬ 
tures  that  included  various  combinations  of  values  of  p  and  patterns 
of  Z. 

In  the  present  model,  the  principal  aspect  of  Z  (other  than  "ample 
variances  of  its  rows  and  sample  correlations  among  rows,  which  are 
important  in  any  regression  situation)  that  proved  important  is  the 
relation  of  its  columns  to  the  characteristic  vectors  of  an  approxi¬ 
mation  to  the  inverse  of  the  variance  matrix  A  [4,  pp .  13-18]  . 

If  the  columns  of  Z  are  linear  combinations  of  K  characteristic 
vectors  of  this  modified  inverse,  then  least-squares  estimates  of  y 
are  best  unbiased  and  tests  of  p  ■  0  (like  those  of  Durbin  and  Watson) 
based  essentially  on  a  Vo,  Neumann  ratio  formed  from  least-squares 
residuals  are  unifowly  most  powerful  against  alternatives  in  the  in¬ 
terval  (0,  1).  Furthermore,  the  characteristic  vectors  are  harmonic 
series,  and  if  :he  K  characteristic  vectors  that  approximate  Z  are  of 
low  frequency,  then  one  f  the  approximations  employed  by  Theil  and 
Nagar  [19]  can  be  shown  to  be  close. 

For  these  reasons  the  Z's  employed  in  three  of  our  structure* 
heve  been  formed  so  that  the  last  three  columns  (the  first  column  con¬ 
sists  entirely  of  onee  in  all  of  our  structure*)  would  be  approximate¬ 
ly  equal  (aae  Appendix  B  for  details)  to  three  characteristic  vectors 
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of  relatively  low  frequency,  thus  insuring  that  the  above  conditions 
approximately  hold.  These  Z's  are  described  as  "smooth"  (S) .  For 
three  other  structures,  called  "rough"  (R)  ,  the  Z-matrices  are  con¬ 
structed  so  that  they  cannot  be  closely  approximated  by  any  K  of  the 
characteristic  vectors.  For  the  remaining  two  structures,  called 
"empirical"  (E) ,  three  rows  of  Z  are  taken  from  observed  time  series 
of  important  economic  variables. 

The  characteristics  of  our  structures  cited  so  far  are  summarized 
in  Table  1, 


Table  1 

CHARACTERISTICS  OF  STRUCTURES 


Structure 

Number 

P 

|  Nature 
of  Z 

L  .. 

Samp le 
Size 

I 

.3 

!  s 

30 

0 

S 

30 

3 

-•? 

S 

100 

4 

1 

R  i 

30 

5 

•3 

R 

100 

6 

0 

R 

100 

7 

.  5 

E 

30 

8 

.  9 

E 

1 _ _ _ _ 

100 

In  all  of  the  structures. 


Simple  sample  correlations  between  columns  of  Z  other  than  the  first 
vary  from  -0.349  to  0.937,  and  sample  variances  of  these  columns  vary 
from  0.46  to  0.75.  These  arrsngemants  inaure  that  the  random  term 


contributes  substantially  to  the  variation  in  the  dependent  variable 


in  all  structures,  while  letting  other  structural  characteristics  vary. 
See  Appendix  B  for  a  more  detailed  account. 

For  comparison  with  each  other  and  with  ordinary  least-squares 
estimates  of  parameters,  the  following  estimators  were  employed. 


MAXIMUM  LIKELIHOOD  (ML) 

For  the  model  defined  in  (1)  and  (2) ,  the  likelihood  function  is 
proportional  to 


<Ky,  P,  v) 


-t/2 2.1/2 
v  (1  -  p  )  exp 


-  ^  (y  -  Zy)’  A_1(y  -  Zy) 


Hildreth  and  Lu  [10]  suggest  one  algorithm  for  maximizing  the  loga¬ 
rithm  of  the  above  function.  Computations  were  originally  performed 
partly  by  graphs  and  partly  by  hand  calculations,  but  a  program  for 
digital  computers  has  subsequently  bean  prepared. +  The  authors  [10] 
showed  that  ML  estimates  are  consistent,  and  Hildreth  [12,  13]  subse¬ 
quently  showed  that  they  are  asymptotically  normal  and  asymptotically 
efficient.  Klein  [14]  suggests  another  algorithm,  and  Fuller  and  Martin 
[8]  develop  an  approximate  procedure.  It  has  also  been  claimed  that 
an  iterative  procedure  suggested  by  Cochrane  and  Orcutt  [3]  converges 
to  ML  estimates.  The  Hildreth-Lu  algorithm  was  used  in  this  study  be¬ 
cause  it  contains  some  safeguw’-ds  against  undetected  multiple  maxima. 

THEIL-NAGAR  (TN) 

These  authors  suggest  a  two-step  procedure  for  estimating  the  pa¬ 
rameters  of  ths  model  in  (1)  and  (2).  Based  on  an  extension  of  a 


The  algorithm  is  described  in  Appendix  D. 
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procedure  they  suggest  for  testing  the  hypothesis  of  serial  indepen¬ 
dence  [19],  f*>ey  obtain  an  estimate  of  the  first-order  autocorrelation 
coefficient,  say  p.  Other  parameters  are  then  estimated  by  applying 
the  classical  least-squares  regression  of 

(yt  '  ^t-l}  °n  (2tl  '  K-l.l* . (ztK  ‘  ~pzt-l,Y?  • 

Their  procedure  for  estimating  p  is  based  on  an  approximate  dis¬ 
tribution  of  the  Von  Neumann  ~atio  obtained  by  fitting  a  8-distribution 
to  approximate  moments,  after  which  p  is  obtained  by  linear  interpola¬ 
tion  : 


where  R  is  the  Von  Neumann  ratio  defined  on  p.  31, 

Since  some  approximation  errors  do  not  disappear  with  increaaing 
sample  size,  the  estimator  Is  not  consistent.  Thus,  for  any  given 
structure,  there  must  be  a  sample  alze  for  which  a  consistent  proce¬ 
dure  (e.e.,  ML  above  or  D  below)  becomes  auperior.  Although  Thell  and 
Nagar  are  uneole  to  evaluate  all  the  approximations  they  employ,  their 
rationalization  is  generally  cogent  and  it  seems  lmpc~tant  to  obtain 
whatever  allies  our  data  contain  about  the  relative  performance  of  this 
estimator  with  typical  sample  sizes. 

APPROXIMATE  SAVES  (AB) 

It  would  have  been  desirable  to  compare  other  estimators  with  the 
mean  of  the  Bayesian  postsrlor  distribution  corresponding  to  s  diffuse 
prior.  Unfortunately,  this  would  have  extended  the  computing  task 
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beyond  what  could  be  contemplated  In  the  present  study.  Instead,  the 
mean  of  an  approximate  prior  suggested  by  Zellner  and  Tlao  [22]  is  used. 
From  (1)  and  (2) 


(3) 


K 

yt-i°  +  ztkYk  '  L  *t-i,kV  +  vt 

k*l  k-1 


2,  3  ....  T 


Each  of  the  nonlinear  terms  y^p  in  (3)  ip  expanded  about  the  ML  esti¬ 
mators,  say  ?k  and  (S ,  as  follows: 

(4)  YkP  -  Yk<5  +  (o  -  e ) Yk  +  (Yk  ~  ?k)(i  , 


where  may  be  read  "is  approximated  hv."  Inserting  (4)  into  (3) 
and  collecting  terms  with  the  same  unknown  parameters  y^  and  p ,  yields 


\  \ 

(5)  yt  -  (S  _  Ykzt  l  k  -  p(ytl  “  '  '?k*t_1 
k  ’  k  ’ 


k 


Tkutk 


6‘t-i,k)  +  vt 


which  is  linear  in  y^  and  p.  The  fitted  least  squares  regression  of 


<*t  - 


Vt-i.k* on  <Vi  -^zt-i,k)*  (Iti 


-  6zt-M}* 


gives  estimates  of  the  coefficients  p  and  y's.  Since  this  estimator 
la  an  adjustment  of  the  ML  estimator,  its  relation  to  the  latter  is 


of  particular  interest. 
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DURBIN  (D) 

Durbin  [ 5 ]  suggests  another  two-step  procedure.  Let  y'  and  z' 

L  t  K. 

be  deviations  from  the  respective  sample  means  of  y  and  z  .  The 

procedure  involves  taking  the  linear  regression  of  y^  on  y|  z^> 

z'  ,  z'  ,  z'  The  resulting  regression  coefficient 

t  K  t  **  l  *  i  t-  1 ,  K 

of  y^  ^  is  its  estimate  of  p,  say  p.  We  then  applv  once  more  the 

least-squares  regression  of  (y^  -  py^  on  (z^  -  Pzt_^  ^)  > 

(z  -  pz  ),  Although  these  estimators  are  consistent  and  asymp- 
tK  t  ~*  I ,  K 

totically  equivalent  to  KL,  there  is  room  for  doubt  about  their  finite 
sample  properties.  Equation  (3)  differs  from  a  standard  linear  model 
in  having  a  lagged  dependent  explanatory  variable.  It  is  also  clear 
that  the  variables  on  the  right  will  be  nearly  multicollinear  in  many 


economic  applications. 
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in.  RESULTS 


PERFORMANCE  OF  THE  ESTIMATORS 


Suninary  Tables 

Tables  2  and  3  summarize  the  principal  calculations.  For  each 
structure  and  each  estimation  procedure,  the  tables  show  the  mean, 
variance,  and  mean  square  error  of  the  300  estimates  for  each  of  the 
six  parameters .  Each  entry  in  a  column  haaded  "Mean"  is  the  simple 
arithmetic  mean  of  the  300  estimates  of  the  parameter  indicated  by 
the  row  label  and  the  structure  indicated  by  the  row  group,  using  the 
estimation  method  indicated  by  the  column  group.  The  "MSF."  (mean 
square  error)  and  the  "Va.r"  (variance)  columns  are  similarly  set  up. 
For  example,  the  calculation  of  the  entry  0.049  in  row  three,  column 
five  of  Table  2  may  be  indicated 


(6) 


300 


r(-2) 

V, 


1 

300 


■  ~(2) 
•:  Vy3 

n«l  n 


V 


where  r~2^  -  the  calculated  variance  of  the  300  Thell-Nagar  estimates 
y3  of  y ^  structure  2, 

Y^  “  the  Theil-Nagar  estimate  of  y^  in  the  ntfl  sample  generated 
n  by  structure  2,  and 

(2' 

H~  '  the  arithmetic  mean,  1.008,  of  these  estimators  (it  ap- 
Y3  peart  in  row  three,  column  4). 

The  corresponding  mean  square  error,  0.049,  in  coluw  six  mav  be  in¬ 
dicated 


P) 


300 


1 

300 


\ 

n«l 


(2)x2 
y3  i  • 
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Tab k-  2 

Sl'MMARY  OF  ESTIMATES  FOR  SAMPLE  SIZE  V' 


St  mcturc 

10  Ho. 

Farnatar 

UtiMtld 

Trua 

Valua 

2 

Ti 

0 

Ti 

1 

Y  3 

1 

1 

0 

I 

1 

EBD 

mt 

-o.ois|o.oj) 

0.0)) 

0.991  Jo .077 

0.077 

1.008  0.0! ^ 

0 .  .'*'0 

l  018  0.078 

0  078 

-■'.120  0.039 

0.05) 

0.83*  0.051 

0.079 

l.vQl  0.07) 

0.07) 

-0.015  0.0))  | 

0.0)) 

)  1 

0.991  0  076 

0.076 

1.008  0.0*9 

0.0*9 

1.018  0.077 

0.078 

I 

-0.062  0.0)*  | 

0.0)8 

0.«5*jo.051 

lo.OH 

1  004  *0.075 

[  0 .075 

1 

— 

Y1 

0 

-0.008 

0.065 

0.065 

2 

1 

0.995 

0.116 

0.117 

Y) 

l 

0.989 

0.060 

0.060 

’* 

1 

1.01) 

0.102 

0.102 

» 

0.  ) 

0.15) 

0.0*1 

0.062 

V 

1 

0.8)5 

0.05) 

0.0*0 

“ 

1 

1- 

1.002 

0.076 

0.076 

i.o»o jo. ■  o  »»7 

o.*)i  o. m  o  m 
l  .OO*  jo  101  j  0 .201 

0 . If* jo. 0*1  !  0.0*5 

i  j 

0.  7*7|0.05*  I  0.0*5 
C  *54  0  071  i  0.0*0 


o  ***jo  0))  !  o  on 

1.005  jo. 01*  j  0.01* 

I  f 

O.M  l| 0.0)1  O.v*0 

0  84)j0  0*1  I  0.087 

i .oil Jo  on  i  o  on 
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(2) 

where  y^  «*  the  true  vslue  of  y^,  1,  in  structure  2. 

Two  sets  of  estimates  of  the  variance,  v,  were  obtained  for  each 
structure-method  combination.  The  first  is  the  quotient  of  the  sum 
of  squares  of  residuals  over  the  number  of  observations.  Empirical 
means,  variances  and  mean  square  errors  for  estimates  calculated  in 
this  way  appear  in  the  upper  rows  labeled  v  in  each  of  the  four  sec¬ 
tions  of  the  tables. 

For  methods  other  than  LS ,  the  second  set  of  estimates  of  v  (fig¬ 
ures  in  parentheses)  are  calculated  by  dividing  each  sum  of  squares 
of  residuals  by  T-5  instead  of  T.  Fitting  5  parameters  to  achieve  a 
low  sum  of  squares  tends  to  make  the  resulting  sum  less  than  that 
which  would  correspond  to  true  values  of  p  and  y.  Since  the  estimates 
are  nonlinear,  one  does  not  know  that  this  is  the  appropriate  adjust¬ 
ment,  but  it  seems  a  reasonable  one  to  try. 

For  LS ,  the  second  set  of  estimates  of  v  are  the  sum  cf  squares 
of  residuals  divided  by  T-4.  This  is  what  someone  who  applied  L„ 
would  ordinarily  use  to  estimate  the  variance.  For  p  0  it  is  known 
to  be  biased,  but  the  bias  could  not  be  computed  without  knowing  the 
true  value  of  p. 

Table  2  includes  the  structures  involving  30  observations,  and 
Table  3  includes  those  with  100  observations  in  each  sample.  The 
structures  in  each  table  are  arranged  in  order  of  increasing  value  of 
P . 

Parc  of  the  information  about  relative  MSEs  in  Tables  2  and  3  is 
presented  more  simply  in  Table  4.  The  first  column  corresponding  to 
each  method  contains  MSE  averages  for  estimates  of  the  four  y's  for 
each  structure  and  for  various  sets  of  structures.  For  instance,  0.059 


AVERAGES  OF  MEAN  SQUARE  ERROR 
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is  the  average  of  four  mean  square  errors  of  y's  estimated  by  the  ML 
method  for  structure  2,  and  0.142  is  the  average  of  these  mean  square 
errors  over  all  structures  with  30  observations  in  each  sample.  For 
methods  other  than  LS  the  second  column  contains  MSEs  of  estimates  of 
p  for  each  structure  and  averages  for  selected  groups  of  structures. 
Each  entry  in  the  third  column  is  a  weighted  average  of  the  correspond¬ 
ing  entries  in  the  first  two  columns  and  represents  the  average  MSE 
for  estimates  of  all  five  coefficients  for  the  indicated  method  and 
structure  (or  group  of  structures).  Mean  square  errors  for  estimates 
of  v  are  excluded  in  Table  4  since  they  depend  on  adjustments  for  fit¬ 
ted  coefficients,  and  the  appropriate  adjustments  for  our  nonlinear 
estimates  are  not  known. 


Comparison  of  the  Varlouc  Estimators  of  p 

To  compare  the  biases  of  the  various  sstimator9,  the  pertinent 
Monte  Carlo  information  was  extracted  from  Tables  2  and  3  and  aunowr- 
lz#d  In  Table  5.  Inspection  of  the  table  leads  to  several  general 
observations . 

Table  5 

MEANS  OF  DIFFERENT  ESTIMATORS®  OF  p 


Structure 

P 

ML 

IN 

D 

. 

[2 

0 

-0.120 

-0.062 

-0.136 

T-30 

1 

0.3 

0.153 

0.194 

0.109 

7 

0.5 

0.298 

0.309 

0.268 

4 

0.7 

0.613 

0.512 

0.552 

[3 

-0.7 

-0  .  . 

-0.672 

-0.694 

T-100 

6 

0 

-0.039 

-0.026 

-0.034 

5 

0.3 

0.270 

0.263 

0.264 

8 

0.9 

0.870 

0.797 

0.822 

*The  AB  estimator  was  axcluded  becauss  it  is  an  ad¬ 
justment  of  the  ML  procedure  and  appears  to  be  numer¬ 
ically  close  to  the  ML  estimators  for  most  samples. 


I 
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A.  When  p  is  nonnegative ,  all  three  estimators  persistently  tend 
to  underestimate  p  on  the  average.  Since  we  have  only  one 
struature  with  negative  p  and  the  three  estimators  come  close 
to  the  true  value ,  we  have  negligible  evidence  for  this  case. 

B.  Judging  by  the  absolute  deviations  of  sample  means  from  their 
respective  true  values,  the  TN  estimator  of  p  looks  slightly 
better  for  samples  with  30  observations  and  relatively  small 
p,  i.e.,  |p|  <  0.3;  however,  ML  appears  to  perform  better  for 
samples  with  100  observations  and  relatively  large  absolute 
values  of  p.  The  Durbin  procedure  appears  to  be  a  little 
better  than  TN  for  samples  with  100  observations ,  but  overall 
it  appears  least  favorable  among  the  three  estimators . 

Since  an  estimator's  performance  depends  not  only  on  the  magnitude 
of  its  average  bias  but  also  on  its  variance,  we  have  computed  the  ra¬ 
tios  of  the  mean  square  errors  of  TN  and  D  estimates  over  those  of  the 
ML  estimates.  The  results,  presented  in  Table  6,  tend  to  confirm  the 
above  observations  regarding  the  relative  performances  of  the  three 
estimators . 


Table  6 

RATIOS  OF  MEAN  SQUARE  ERRORS  OF  TN  AND  D  ESTIMATORS  OF  p  TO  MI.  ESTIMATOR 


Structure  p 

MSE  of  TN  estimate  of  p 

MSE  of  D  estimate  of  p 

MSE  of  ML  estimate  of  p 

MSE  of  ML  estimate  of  p 

"2  0 

0.717 

1.038 

in 

1  0.3 

0.742 

1.258 

7  0.5 

0.819 

1.217 

L*  0.7 

1.550 

1.450 

*3  -0.7 

1.167 

1.000 

T-100 

6  0 

0.909 

1.000 

5  0.3 

0.923 

0.923 

L8  0.S 

2.667 

1.833 

The  MSE  of  the  TN  estimator  of  p  is  smaller  than  that  of  the 
ML  estimator  for  smples  with  relatively  small  p  and  with  30 
observations .  The  ML  estimator,  however,  has  a  smaller  MSF 
for  samples  with  relatively  large  p  and  100  observations .  Re¬ 
ferring  back  to  Tables  2  and  3,  one  sees  that  TN  variances  arc 
consistently  smaller  than  ML  vainanoe  for  sample  sise  30  and 
slightly  smaller  tn  two  cases  with  sample  sire  100.  The  gen¬ 
erally  smaller  MSEs  for  ML  estimates  with  100  observations 
ore  therefore  due  to  smaller  biases. 


D.  The  D  estimator  seems  inferior  to  the  ether  two  estimators 
in  terms  of  the  mean  square  error  ratios. 

Comparison  of  the  Various  Estimators  of  y's 

E.  The  sample  means  of  all  the  estimators  of  y's  are  similar  and 
are  close  to  their  true  values.  The  LS  estimator  is  known  to 
be  unbiased.  The  other  estimator ?  also  seem  to  shew  very  lit¬ 
tle  bias  even  with  a  ZO -observation  sample. 

To  examine  the  relative  efficiency  of  the  various  y  estimators, 
we  divided  the  average  mean  square  error  of  y's  for  each  of  the  three 
estimators,  ML,  TN  and  D,  by  that  of  the  LS  estimator.  This  gives  some 
indication  of  what  an  investigator  will  gain  if  he  uses  one  of  the  more 
complicated  methods  instead  of  the  ordinary  least  squares  u’thod.  Re- 
suits  are  presented  in  Table  7.  For  instance,  the  first  entry  in  the 
table,  1.035,  was  obtained  by  dividing  0.059  by  0.057.  These  mean 
square  average  errors  fer  y's  are  given  in  Table  4. 


Table  7 

RELATIVE  EFFICIENCY  OF  DIFFERENT  ESTIMATORS 
OF  y's  COMPARED  TO  LS  ESTIMATOR 


Structure 

_p 

ML/LS 

TN/LS 

D/LS 

~2 

0 

1.035 

1.035 

1.053 

T-30 

1 

0.3 

1.012 

1.000 

1.094 

7 

0.5 

0.869 

0.843 

0.895 

_4 

0.7 

0.707 

0.695 

0.820 

"3 

-0.7 

0.750 

0.750 

0.750 

T-100 

6 

0 

1.000 

1.000 

1.167 

5 

0.3 

0.905 

0.905 

1.048 

L0J_ 

0.540 

0.448 

2.700 

F.  Judging  by  the  relative  efficiency ,  TN  is  a  little  better 
than  ML  for  earrplee  of  site  Z0  and  about  the  scene  for  sam¬ 
ples  of  site  100.  The  D  estimator  performs  slightly  worse 
than  both  the  ML  and  TN  estimators . 

G.  For  samples  with  only  Z0  observations  and  relatively  small  value 
of  p  (0.Z),  the  other  three  estimators  do  not  have  advantages  over 
the  LS  estimator. 
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Interpretation  of  comparisons  for  groups  of  structures  is  compli¬ 
cated  because  the  number  of  structures  that  could  be  investigated  pre¬ 
vented  construction  of  a  balanced  design  ref lecting  all  of  the  prop¬ 
erties  considered  important.  Thus  the  three  structures  with  smooth 
Z's  include  two  with  T  ■  30  and  one  with  T  -  100,  while  the  three  with 
rough  Z's  include  two  larger  samples  and  one  smaller  one.  Comparison 
of  the  smooth  and  rough  rows,  therefore,  indicates  only  that  the  ef¬ 
fect  of  smoothness  in  Table  4  is  small  relative  to  sample  size  in  our 
experiment . 

A  little  better  hint  can  be  obtained  by  averaging  MSEs  for  struc¬ 
tures  1  and  3  and  comparing  these  with  averages  for  4  and  5;  the  re¬ 
sults  are  shown  in  Table  8. 

Table  8 

AVERACE  MSE  FOR  ALL  COEFFICIENTS  AND  SELECTED  STRUCTURES 


Structure 

Combination 

ML 

TN 

LS 

AB 

D 

1,  3  (S) 

0.043 

0.042 

0.04  7 

0.048 

0.048 

4,  5  (R) 

0.060 

0.061 

0.094 

0.081 

0.071 

The  comparisons  in  Table  8  are  a  little  more  meaningful  than  the  S  and 
R  rows  of  Table  4,  since  each  row  of  Table  8  refers  to  a  pair  of  struc- 
tuxas  with  T  values  30  and  100  and  |p|  equal  to  3  and  7  (see  the  de¬ 
scription  of  structures  in  Table  1).  This  ahowm  a  tendency  for  lcwer 
MSE  with  smooth  independent  variables,  particularly  for  LS.  It  is 
clear,  however,  that  any  conclusions  on  effect  of  smoothness  based  on 
data  from  the  present  study  would  be  very  tenuous.  This  should  be  in¬ 
vestigated  further  analytically  and,  if  necessary,  by  Monte  Carlo 
trials  specifically  designed  for  this  purpose. 
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AB  estimates  are  obtained  by  adjusting  ML  estimates.  From  Tables 
2  through  4,  It  appears  that,  on  the  average,  the  adjustment  worsens 
the  estimates,  at  least  when  judged  by  KSE.  It  Is  also  of  Interest  to 
know  whether  or  not  the  adjustment  Is  typically  large  or  small.  That 
It  was  less  than  0.0S  in  most  cases  in  the  present  study  is  Indicated 
by  Table  9,  which  contains  frequencies  of  the  differences  in  AB  and 
ML  estimates  of  p  and  of  y^. 


Table  9 

COMPARISON  OF  MAXIMUM  LIKELIHOOD  ESTIMATES  WITH  APPROXIMATE 

BAYES  ESTIMATES 


it  should  be  noted  that  the  above  observations  and  others  to  fol¬ 
low  are.  In  the  first  Instance,  descriptive  statements  of  how  certain 


ym&tftgp- 


/ 


statistics  behaved  in  this  one  experiment.  Since  300  samples  were 
drawn  for  each  structure,  we  hope  that  the  observed  characteristics 
are  generally  representative  of  these  structures.  It  is  unknown  how 
well  these  structures  represent  those  commonly  encountered  in  practice 
and  how  many  of  the  properties  we  have  noted  will  hold  for  different 
structures.  Thus  it  is  desirable  that  hints  furnished  by  these  studies 
be  supplanted  by  precise  analytical  results  as  quickly  and  completely 
as  possible.  For  Important  properties  that  remain  intractable,  further 
Monte  Carlo  experiments  with  different  structures  are  in  order. 

ML,  TN  and  D  seem  to  understate  p  svstematically  (at  least  for 
nonnegative  p)  ,  suggesting  that  a  systematic  adjustment  in  each  esti¬ 
mator  might  improve  its  accuracy,  especially  for  small  samples.  This 
seems  worth  pursuing,  but  the  authors  believe  that  further  analysis  of 
the  distributions  of  the  two  estimators  is  in  order  before  recommenda¬ 
tions  are  formulated.  For  the  ML  estimator,  the  matter  is  discussed 
a  little  further  in  connection  with  the  discussion  on  tests  of  good¬ 
ness  of  fit. 

The  tendency  for  maximum  likelihood  to  give  better  estimates  than 
alternative  procedures  when  |p|  is  large  is  confirmed  bv  a  studv  con¬ 
ducted  independently  by  David  F.  Reilly  [16], 


TSSTS  OF  SIGNIFICANCE 

Although  this  study's  emphasis  is  on  estimator  performance,  it 
would  have  been  wasteful  not  to  have  used  the  data  generated  tc  check 
the  behavior  of  commonly  used  tests  of  significance  at.  well.  Accord¬ 
ingly,  Tables  10  and  11  present  the  fraction  of  samples  that,  for  each 
of  several  teeta,  rejects  the  null  hypothesis  p  -  0  (one-sidsd)  or 
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Table  11 

HONTE  CARLO  APPROXIMATION  TO  PROBABILITIES  THAT  VARIOUS  TESTS 


I 

*H  O 
^  -H 
4)  O  4J 
^  O  03 
-H  X  iX 


(0  LJ 

V  c 

H  4> 

a 

±J  s 


T-l  O 

H  tl  -rt 
4)  O  U 
J<  O  <V 

x:  ot. 

J 


1.000 

0.008 

0.450 

1.000 

jO  ^ 

- 1 

O 

lO  r-4  If)  l 

1  o 

0  0^1 

1  o 

•  *  • 

• 

•H  O  O 

rH 

>\  /*>  /-N 

/— *\ 

-o  >j  n 

o 

i  O  ci  X)  j 

1  O  i 

io  O  CN  1 

1  °  i 

1  *  •  • 

o  o  o 

O  ! 

' — '  l. _ < 

w! 

O  ci  ci 

o  i 

O  O  O  1 

1  O  I 

O  O  m  i 

1  o 

•  •  • 

•  : 

rH 

O  N  ^ 

o 

O  O  lO  1 

1  O  I 

O  O  in  1 

1  o 

•  •  • 

• 

o  o 

^  i 

OO  CD  N 


;  — 1  <n  cn  o 

i  o  O  —  cm 


o  o  o  o 


oo<t® 
cl  D  o  -1 

I  O  OM  CO  I 


<t  x>  ao  <t 

c^i  m  «-h  oo 
<N  <?  I 


o  o  o  o 


T'  N  N  if) 

O  CN  O  o 

•  O  o  — 1  m 


o  o  o  o 


'  O  o  00 

)  i  H  J  H  OC  | 

f.  I  O  O  (M  \C  I 


o  o  o  o 


n  m  n  j< 


o  o  o  o 


o  n 

O  <*>  v£> 
O  O  vO 


O  M  O 
O  N  H  I 

O  O  CT  I 


O 

O  s  «  l 
©  O  -c  I 


O  O  c> 

O  CN  \D  I 


o  o  \o  i 
<-*  o  o 


o 

® 

o 

O 

•  i 

4) 

o 

O 

f-J 

08 

r-H 

*-H 

'w' 

U 

4) 

4) 

o 

rH 

N 

N 

o 

4-t 

4) 

tH 

o 

0 

> 

cn 

ao 

**H 

C 

4> 

c-H 

U-< 

O  •"  o  o 

Cl  Cl  CM  |  |  o 

O  O  OO  I  |  O 


<r  cm  q 
lA  O'  i/i  c\ 
l  O  O  CM  00  I 


o  o  o  o 


-J  O  00  hH 
n  ■«  Q  - 
"  1  O'  I 


o  o  o  O  I 

I 

! 


;  o  cm 

!  “D  A  N  o 

’  i  M  ci  4  n  i 


o  o  o  o 


Cm  t-'.  ,-"i  r~» 

— *  X>  o  m 

I  O  O  <N  sO  I 


o  o  o  o 


O  r»  o  n. 

I  IT!  1/1  | 

t  O  N  'J  CD  I 


o  o  o  o 


m  in  ai 


io  o  o  o  o! 

t - -1 


&  sr\ 

*4  hi  n  ^  oo 


x>  u 
o 

lm  4  n  sj  co 
<N  -4 


P  C  08 

U  0)  (V 

«D  U  rH 

V*  u  Q. 

^  V  g 

P-  a 
4)  I  CD 

X  CN 


jc  x: 

\0  lO 


3  **  m  oo 

a  u 

c.  o  -a 


£*.  s  s 

« * 

S  o  o 


i  i 


c  m 

m  jc 

jc 

*  H  cm 

ki 

*  *i 


5«  3 

>  jj 
C  -H  O 

*  3  2 

•C  ^  u 

H  u  vi 

•  C  JD 

o 

u 

c 


I 


-23- 


p  -  0  (two-sided)  for  each  structure.  Each  entry  is  the  fraction  of 
300  samples  In  which  the  indicated  test  rejected  the  null  hypothesis 
in  question.  The  result  of  the  Durbin-Watson  test  is  sometimes  incon¬ 
clusive  (see  [6],  p.  409).  The  proportion  of  cases  in  which  this  oc¬ 
curred  is  indicated  in  parentheses  beside  the  entry  Indicating  the  nro- 
portion  in  which  the  null  hypothesis  was  rejected. 

The  Von  Neumann  ratio  test  [20],  the  Theil-Nagar  test  [19],  and 
the  Durbin-Watson  test  [6,  7]  have  frequently  been  used  in  econometrics. 
Ail  are  based  on  the  Von  Neumann  ratio  of  mean  successive  difference 
to  sample  variance.  An  investigator  using  likelihood  methods  would 
find  f,  the  ML  estir  a  of  p,  or  the  likelihood  ratio  a  natural  test 
statistic . 

Hildreth  shows  [12]  that  fi  is  asymptotically  normally  distributed 

1-D2 

with  mean  p  and  variance  Hence,  a  test  based  on  this  asymptotic 

distribution  may  be  applied  to  these  null  hypotheses  bv  referring  to 
a  normal  distribution  with  zero  mean  and  variance  1/T. 

The  likelihood  ratio  test  has  teen  applied  bv  assuming  that 

2 

-2  log  \  (where  1  is  the  likelihood  ratio)  is  approximately  The 

likelihood  ratio  is,  of  course,  only  useful  for  two-tailed  tests. 

Since  the  Durbin  and  Watson  tables  do  not  provide  for  a  1-percent 
tvo-tailed  test,  the  results  shown  for  the  tvo-tsilsd  DW  test  sre  for 
an  intended  2-percent  elgnificanca  level.  Theil  end  Hager  did  not  rec¬ 
ommend  that  two-tailed  tasta  ba  performed  using  thslr  tabic;  but,  be¬ 
cause  their  tabulated  critical  point#  are  almost  idsntical  to  the 
critical  points  d^  in  the  Durbin-Watson  tables  (sss  rssult  D  bslow) , 
one  could  obtain  the  results  for  two-tsilsd  TN  tests  by  adding  the 

i 

The  refinement  of  the  Theil-Nagar  te»t  suggested  by  Hanahaw  |9] 
came  to  our  attention  after  computation!  were  under  way. 
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regular  entry  in  the  DW  column  to  the  parenthetical  entry  immediately 
to  the  right. 

Principal  results  indicated  by  Table  8  are  the  following: 

A.  There  were  many  inconclusive  applications  of  DW,  as  previous¬ 
ly  noted  by  both  theorists  and  practical  workers. 

3.  The  low  empirical  significance  levels  associated  with  one- 
tailed  £  tes*s  when  p  is  actually  zero,  and  the  high  levels 
for  two-tai  led  tests,  are  consistent  with  the  te>ulencu  pre¬ 
viously  noted  for  <5  to  be  negative  when  p  =  0.  This  suggests 
that  tests  based  on  6  cannot  be  recommended  for  moderate¬ 
sized  samples  until  a  better  approximation  to  its  distribu¬ 
tion  t3  developed. 

For  3arrples  of  size  20,  the  tabulate  i  power  of  the  test 
•  be  discounted  because  the  test  rejects  a  true  null  hy¬ 
pothesis  much  more  frequently  than  it  should. 

D.  A  comparison  of  the  Tit  and  DW  columns  for  one-tailed  tests 
indicates  that  the  proportion  rejected  by  Til  is  equal  ( with¬ 
in  rounding  error )  to  the  proportion  rejected  by  DW  plus  the 
proportion  inconclusive  by  DW.  Inspection  of  their  tables 
indicates  that  the  TN  critical  values  are  within  0.01  of  the 
corresponding  upper  DW  critical  values  exoj.pt  for  samples 
smaller  than  20.  Thus,  in  practice,  applying  IN  is  virtually 
the  same  as  applying  DW  and  rejecting  the  null  hypothests  if 
the  DW  procedure  either  indicates  rejection  or  is  inconclu¬ 
sive. 

K,  The  LR  test  based  on  the  asymptotic  distribution  is  not  very 
powerful  for  samples  of  size  3  and,  for  samples  of  size  100, 
the  rejection  rate  for  true  hypotheses  is  lower  than  the  in¬ 
tended  significance  level. 


approximate:  distributions 

As  mentioned  in  Sec.  I,  Hildreth  [13J  has  shown  that  the  ML  es¬ 
timators  are  asymptotically  distributed  according  to  a  multivariate 
normal  law  with  ?,  6,  0  mutually  asymptotically  independent.  The 

■fc 

•asymptotic  variances  are 


Limits  of  these  moments  are  nown  to  equal  the  corresponding 
momenta  of  the  limiting  distribution,  since  it  can  be  shown  that  fourth 
moments  of  the  ML  estimators  are  bounded.  Sae  [13],  p.  10. 
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(8)  lira  E*?"  (y  -  y)(f  -  y)  ’  vV 

X-KK 

lira  e/F  (0  -  p)2  -  (1  -  p2) 

T>“ 

liffl  E/f  (v  -  v)2  -  2v2  , 

J->OD 

where  y>  P»  v  are  the  ML  estimates  and 

'  I  -1  N-1 
V  -  i^lin  -  Z'A  Zy  . 

T-** 

It  was  conjectured  that,  for  many  purposes,  the  asymptotic  distribu¬ 
tion  of  f  would  prove  a  tolerable  approximation  in  samples  of  the  size 
often  encountered  in  econometric  studies,  but  that  for  £  and  0  the 

asymptotic  distributions  would  be  less  satisfactory. 

2 

The  x  goodness-of-fit  statistics  listed  in  Tables  12,  13,  and 
14  tend  to  confirm  this  conjecture.  Table  12  was  constructed  by  de¬ 
termining  13  intervals  for  each  component  of  y  end  computing  the  ex¬ 
pected  frequency  of  estimates  in  each  interval  under  the  assumption 
that  the  eatimator  was  distributed  according  to  its  asymptotic  law. 
Adjacent  Intervals  with  small  expected  frequenciss  ware  combined  to 
follow  Cochrane's  recommendation  that  no  more  than  20  percent  of  the 
remaining  intervals  should  have  expected  frequencies  smaller  then  5. 

This  determined  the  "df"  entries. 

Observed  frequencies  in  each  interval  were  then  tabulated  and  a 

2 

X  value  for  each  estimator  was  computed  by  the  familiar  one-way  formula, 

2  \  ‘'I’0/ 

x  •  L  - 5 -  » 

i«i  i 


(9) 
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Table  12 

\2  STATISTICS  FOR  ASYMPTOTIC  DISTRIBUTION  OF  y's 


lx 

'2 

r  s 

X, 

2 

: 

) 

m 

HI 

- - 

St  roc  tu  re 

d  f- 

57.  Po l n  t  x 

r 

.1.  t  . 

37.  Points 

37.  Points 

A 

r: 

* 

13 . 3 

in.  9 

10 

18.  3 

n.h 

13.3 

)  7 

18.  3 

11.7 

r*3<> 

i 

10 

18.  1 

7  .  a 

10 

18.  ) 

i.h 

15  5 

10.  7 

18.  > 

in.  ? 

; 

5'* 

13.  ) 

3.9 

1  i 

19  ? 

m.o 

i  i 

19.  7 

15.2 

H  IE 

lb  9 

3.8 

.  ■* 

11 

39.  7 

12.9 

8 

15  5 

•  2 

8 

13.3 

a  .  0 

8 

13.3 

11* 

"  3 

_ 

9.  3 

b .  8 

b 

12  b 

2 .  ii 

mm 

1  2  .  b 

1  .  1 

!  h 

12  b 

5.9 

6 

6 

13.  b 

?  . 

b 

12.  b 

8.  6 

■S 

12.6 

1  5 

b 

12  b 

8  3 

5 

b 

12.6 

8.8 

b 

12.6 

5 .  b 

■fl 

12.6 

6  .  b 

b 

12  b 

9  3 

.d 

LJIL 

18.  > 

1  *.b 

9 

16.9 

/ .  n 

10 

18.  3 

13.2 

8 

15  5 

L_i_L 

where  0^,  are  respectively  the  observed  and  expected  frequencies, 
and  the  number  of  Intervals  is  I. 

Table  13  was  constructed  similarly  except  that  alternative  theo¬ 
retical  distributions  were  used  to  determine  expected  frequencies  in 
2 

calculating  the  /  statistics  appearing  in  the  last  two  columns.  For 

* 

the  column  headed  8  ,  a  modified  8-  distribution  was  determined  by 


‘Let  f  (x)  - - I—  xP'  1  (1  -  MO'1  for  0  ••  x  ■  l  bo  a  3-densHv  and  li-t  w  -  2x  -  1.  Then 

3  Bfp.l) 


K(w) 


2*>+q'1  B(p,q) 


; !  ♦  w>p‘ 1  (1  -  w) 


q-  1 


for  -1  *  w  <  I  »s  the  density  of  w 


Ew 


PS-3. 

P  +  q  ' 


V.«r  (w) 


_ Jtk 13 _ 

> 

( p  +  q )  “  ( i>  *■  q  ♦  l ) 


Setting  Ew  *  0 ,  Var  w 


y it  Ids 


I 


P  B 


q  » 

IlL—tl 

2 

I  (1  *  3).  var 

1  -  P 2  9 

7 

(1  *  p) 

yle Ids 

r  a2bt2 

♦  A1 

„ .  i 

r  .. .  A,v _ 3i 

[m  *  orrn--' 

Tnrv  Aj  • 

q  2 

L'l  ♦  P )  [  T  ( 1  -  ♦  V'  J 

nlh  A  *  H  ♦  0  -  j  -  ,  B  -  (1  -  f>  ♦  l  *  -£) 


T  T 
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transf orming  the  variable  so  that  the  interval  of  nonnegative  density 
was  (-1,  .1)  rather  than  (0,  1),  and  then  determining  the  remaining 
free  parameters  to  make  the  mean  and  variance  equal  to  their  asymptotic 
values  ,  p  and  ^-P  . 


Table  13 

X2  STATISTICS  FOR  p 


Structure 

P 

d.  f. 

57,  Points 

Calculatec 

2 

Values  of  x 

Asymptotic 

A 

6 

AA 

6 

"2 

0 

8 

15.5 

196.7 

194.23 

8.95 

T=30 

1 

0.3 

8 

15.5 

317.9 

277.56 

5.12 

7 

0.5 

8 

15.5 

945.6 

686.57 

28.28 

L> 

0.7 

6 

12.6 

70.6 

194.69 

103.79 

"3 

-0.7 

4 

9.49 

1.8 

4.05 

11.40 

c 

o 

*»-< 

II 

H 

6 

0 

6 

12.6 

52.3 

51.78 

10. 10 

5 

0.3 

6 

12.6 

40.7 

37.76 

5.08 

|_8 

0.9 

4 

9.49 

131.8 

131.89 

149.29 

The  theoretical  distribution  uoed  in  calculating  tha  column 

AA  A 

headed  6  wag  similar  to  that  for  8  accept  that  the  mean  was  set 

2 

equal  to  p  -  — ^  ^  P -  and  the  variance  equal  to  —  +  (1  +  p)- 

X 

The  latter  expressions  crudely  approximate  me  means  and  variances 

that  appear  in  Tables  2  and  3  for  various  values  of  p.  Tha  8*  and  8** 

distributions  wars  superficial  guesses  made  in  a  quick  attempt  to  find 

a  better  approximation  to  the  distribution  of  p  in  typical  samples. 

** 

Though  8  does  reduce  the  'badness"  of  fit  substantially,  except  for 

high  values  of  p,  it  does  net  look  promising  to  us,  and  we  believe  an  * 

attempt  to  determine  more  properties  of  the  finite-sample  distribution 

of  £  analytically  should  precede  further  attempts  to  find  a  better  ap¬ 
proximation. 
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Table  14  also  contains  x  values  calculated  from  the  asymptotic 
distribution  (this  time  of  O)  and  another  which,  it  wa9  guessed,  might 
provide  a  better  fit  for  typical  samples.  The  asymptotic  distribu¬ 
tion  is  normal  with  mean  1  and  variance  2/T;  the  alternative  was  ob¬ 
tained  by  assuming  that  Tvi  was  x~  v  ,  ■  The  latter  amounts  to  treat-  1 

1 —  K.“  JL  | 

ing  o  as  though  it  entered  linearly.  Though  this  approximation  did  j 

fit  well  for  samples  of  size  30,  neither  it  nor  the  asymptotic  dis-  j 

tribution  was  a  good  approximation  for  samples  of  size  100.  Here,  j 

again,  closer  study  of  properties  of  the  actual  finite  sample  distri-  i 


bution  is  in  order. 

Table  14 

X2  STATISTICS  FOR  0 


Structure 

Asymptotic 

Gamma 

d.  f. 

37.  Points 

2 

X 

d.f. 

57,  Points 

2 

X 

"2 

10 

18.3 

157.48 

— 

9 

16.9 

9.94 

1 

10 

18.3 

162.74 

9 

16.9 

4.12 

7 

10 

18.3 

246  43 

9 

16.9 

19.  78 

u 

10 

18.3 

158.66 

9 

16.9 

6.56 

■3 

6 

12.6 

41.24 

7 

14.1 

27.97 

T*I00 

6 

6 

12.6 

51.55 

7 

14.1 

168.25 

5 

6 

12.6 

33.84 

7 

14  1 

13.70 

.8 

6 

12.6 

13.04 

7 

14.1 

37.90 

One  reason  for  examining  the  fit  of  approximations  to  the  maxi¬ 
mum  likelihood  estimators  is  the  conjecture  that  It  may  be  possible 
to  construct  a  useful  approximate  Bayesian  procedure  for  applications 
of  this  model  if  a  sufficiently  simple  and  accurate  approximation  can 
be  found.  Prospects  for  such  a  procedure  are  enhanced  if  the  esti- 


1 

J 

! 

j 

» 


V 

\ 

I 


I 


Se«  pp.  426-427  of  [11], 
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mators  y ,  p,  0  are  "approximately"  independent  in  samples  encountered 
in  practice.  The  aspect  of  independence  that  can  most  readily  be 
checked  (and  is  quite  possibly  the  most  import an ^  «ns;ect  if  utility 
functions  are  approximately  linear)  is  linear  noncorrelation. 

For  this  prospect,  Table  15  is  highly  encouraging.  Simple  corre¬ 
lation  coefficients  between  p,  v,  and  components  of  y  are  presented 
for  each  structure. 


Table  15 

SIMPLE  CORRELATION  COEFFICIENTS 


Pairs  of 

Structure 

s 

Estimators 

1 

2 

— 

3 

4 

5 

6 

7 

8 

a 

0.084 

-0.004 

0.  i05 

-0.049 

-0.056 

0  003 

0.076 

-0.076 

n 

»  >2 

-0.096 

0.024 

0.005 

-0.051 

0.008 

0.070 

0.036 

-0.072 

e  V3 

-0.047 

0.060 

0 

0.048 

0.005 

0  045 

-0.031 

0  008 

5  >4 

-0.045 

0.058 

0.  104 

0.084 

0.082 

-0.003 

-0.061 

-0. 128 

A  /V 

v  Yl 

0.078 

-0.023 

0.0C7 

-0. 100 

0.037 

-0.025 

0 

-0. 156 

y2 

-0.028 

0.028 

0.026 

0.073 

-0  053 

0 

0.016 

-0.038 

v  Y3 

0.  109 

0.045 

-0.043 

-0.104 

0.053 

0  025 

0.007 

0.001 

*  *4 

-0.045 

-0.031 

-0.082 

-0.027 

-0.043 

0.087 

-0.019 

0.322 

v  0 

0.096 

0.  177 

-0.067 

0.  186 

— 

0.  142 

-0  001 

0.297 

0.  195 

For  300  observations,  the  significance  points  of  the  sampling  dls- 
tributien  of  simple  correlatien  coefficients  under  the  assumptions  of 
normality  and  p  -  0  are  +  0.1133  at  the  5-percent  level.  Consequently, 
among  the  61  correlation  coefficient*  examined,  only  7  rejected  the 
null  hypothesis.  Since  5  out  of  the  7  rejected  cases  involve  corre¬ 
lation  coefficients  between  p  and  0,  ve  probably  cannot  assume  that 
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they  are  independent  In  moat  samples  encountered  in  practice;  however, 
the  elements  of  y  are  approximately  uncorrelated  with  p  and  v. 
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Appendix  A 

METHOD  FOR  GENERATING  ARTIFICIAL  DATA 


Our  procedure  for  generating  time  series  with  known  properties 
consisted  of  the  following  steps  : 

a.  We  first  decided  on  a  particular  combination  of  parameter 
values  for  the  model  represented  by  (1)  and  (2).  All  together, 
eig.t  different  combinations  of  the  parameter  values  were  con¬ 
sidered  (see  Table  1,  p.  5). 

b.  The  values  assumed  by  all  the  explanatory  variables,  Z  ,  for 
t  =  1,,..,  T  and  k  =  l,...,  K,  were  also  specified.  In 
general,  these  values  were  varied  from  one  structure  to  another. 

c.  We  then  generated  T  random  numbers,  each  of  which  was  normally 
and  independently  distributed  with  mean  0  and  variance  1. 

d.  These  random  numbers  were  used  as  independent  disturbances  of 
the  model  and  T  obsei vat  ions  were  obtained  on  the  dependent 
variables,  conditioned  on  the  assumed  values  of  the  parameters 
and  the  explanatory  variables.  The  yt'8  thus  generated,  to¬ 
gether  with  the  corresponding  Z^'a,  constituted  a  sample. 

e.  The  four  estimating  methods  (ML,  TN ,  AB ,  and  D) ,  plus  the  least 
squares  (LS)  method,  were  each  applied  to  the  above  sample  for 
estimating  y^'s,  v>  P  *nd  the  V°n  Neumann  ratio  statistic  R. 


t-1 


where  u  is  the  LS  estimate 
t 

of  the  disturbance  u  . 

t 


f.  For  each  of  the  eight  structures,  step:,  (c;  through  (e)  were 
repeated  300  times,  The  resulting  300  sets  of  parameter 
estimates  became  the  basic  data  for  our  sampling  experiment 

o- 

with  respect  to  that  structure. 

Procedures  for  generating  data  as  described  above  were  programmed 
in  FORTRAN  IV.  For  those  interested  in  further  experimentation  using 
samples  with  different  characteristics,  usage  of  the  program  is  de¬ 
scribed  in  Appendix  D. 

It  took  approximately  25  minutes  of  IBM  7044  computer  time  to 
obtain  300  sets  of  parameter  estimates. 
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Appendix  B 

SPECIFICATIONS  AND  PROPERTIES  OF  ^'_s 

For  reasons  discussed  in  Sec.  II,  we  constructed  three  different 
types  of  independent  variables.  The  values  of  the  Independent  vari¬ 
ables  in  structures  1,  2,  and  3  are  such  that  they  approximate  the  con¬ 
ditions  favorable  to  TN  and  LS :  those  for  structures  4,  5,  and  6  do 
not.  The  independent  variables  for  structures  7  and  8  were  based  on 
empirical  time  series. 

To  specify  the  independent  variables  of  structures  1  to  6,  let 
us  define  a  typical  element  in  the  Jth  characteristic  vector  of  an 
approximation  to  the  inverse  of  the  variance  matrix  A  as  [4,  p.  17] 

(11)  R(j  ,  t)  -  cos  ^2-  —  -  jirj  t-1,2,  .  . .  ,  T  . 

Using  the  above  notation,  the  independent  variables  for  structures  1 
to  6  are  presented  in  Table  16. 

Characteristics  of  the  assumed  values  of  the  independent  vari¬ 
ables  for  various  structures  are  summarized  in  Table  17.  Note  that 
the  sample  variar.'-es  of  all  the  Z  variables  are  less  than  1,  and  that 
their  sample  correlation  coefficients  are  small  except  for  those  based 
on  the  empirical  data. 
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Tab  1,  It 

SPECIFICATIONS  OF  THE  7,  MATRICES  FOR  STRl’CTCRES  1  TO  h 


Independent 

Variables 

Structure 

1 

r instant  Ter* 

2* 

3 

4 

1 

R(2.t)  + 

P'll.i)  *  c 

R( *> ,  t )  *  t( 

2 

R<0,t) 

R(2,t)  +  t( 

K(  1 1 ,  t  )  ♦  r 

t 

R(S,t)  ♦  r 

3 

R(0,t) 

R(i.t)  + 

R( 23 , t )  ♦  C[ 

R(lU.t)  *  r 

i  r 

\  r 

4 

R(0,t) 

2  L  R()'° 

j  '  R().t’ 

2  L  R(''° 

*-J, 

]•  J  , 

J  1 

2 

J  -  (1,.,  3,  IS,  20, 20 

J2  ■  2  0,11.17,21,26  ■ 

J  ,  «=  '  ), 6,11,  18, 2), 27; 

i  r 

l  c 

1  r 

5 

R(O.t) 

2  l 

2  l  R(l>° 

j  R(l.t) 

J'J2 

’•J) 

J  -  !  1,4.8,  13,20.60' 

|  1 

J2  -  ,2. i, 11,  17,21,72  ’ 

1 )  »  1.6.13. 18,21,83 

0 

Satne  as  Structure  3 

MOTE:  A  rsndo®  element  wii  added  to  each  of  the  independent  variables  (other  than  the  constant 
ter*)  In  the  first  three  structures.  For  Structuces  7  and  8,  fne  original  data  were  taken  fro* 
three  sets  of  empirical  t i*e  series  fro*  U.S.  Statistical  Abstract.  wholesale  price  index; 

(2)  numbers  of  imigrants;  (3)  exports  of  foodstuff.  The  values  of  these  independent  variables 
had  been  adjusted  so  that  their  sa*ple  variances  would  be  0.73,  which  is  saali  relative  to  the 
variance  of  the  rando*  disturbances.  This  was  intended  for  easier  interpretat ion  of  the  saaplmg 
experiment  results. 

4  Is  a  normal  deviate  with  scan  0  and  variance 


T.ibK  !  7 

CHARACTERISTIC1’  OF  Z  VARIABLES:  MEANS,  VARIANCES ,  AND 
CORRELATION  COEFFICIENTS 
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Appendix  C 

MAXIMIZING  THE  LIKELIHOOD  FUNCTION 

The  likelihood  function  of  this  study  differs  from  the  one  devel¬ 
oped  in  w 1 0 J  only  in  the  added  assumption  that  the  in  Eq  .  (1)  ha^ 

4- 

a  stationary  distribution.  In  particular,  it  is  assumed  that  all  of 
the  u^'s  are  normally  distri  ted  with  zero  means  and  equal  variances 
Ll/d  -  o2)]  u. 

To  obtain  ML  estimates  of  \  and  c  the  following  procedure  was 
used:  Assumt  a  particular  value  of  o,  and  compute  the  corresponding 

estimates  of  y  and  the  sum  of  squared  residuals  from  (12)  and  (13), 
respect ive  ly : 

(12)  7(c)  -  *Z  A^Z]'1  rZ  A'V 

(13)  S(p)  =  [y  -  Z>'(c)V  A'1  r y  -  ZV(P)1  • 

This  is  done  for  a  number  of  selected  values  of  c,  and  the  value 
which  approximately  minimizes  S ( o ,  is  determined  to  a  desired  accuracy. 
If  the  value,  o  ,  of  c  that  minimizes  S(c)  can  be  found,  then  5  and 
y  ’  >(:?)  are  ML  estimates  of  o  and  y. 

A  computer  program  was  developed  for  numerical  search  of  a  minimum 
point  of  Lfo)  in  the  interval  -1  -ip  <  1.  Since  S(c)  is  a  'a'.vnomial 
of  high  degrees  in  o,  a  procedure  was  provided  for  safeguarding  against 
multiple  minima.  This  was  done  by  examining  the  successive  first- 
differences  of  S (o)  evaluated  in  the  above  interval  at  an  increment  of 

The  model  based  on  this  assumption  of  stat  iotvarit  v  is  discussed 
in  some  detail  in  Append.:  A  of  1 0 j  and  ;  13j ,  op.  2-8. 
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0.01.  If  the  values  of  these  first  differences  should  change  sign 
more  than  once,  we  would  conclude  that  the  function  probably  did  not 
have  a  unique  minimum.  It  should  be  noted  that  no  mu itiple-minima 

lation  appeared  to  exist  in  all  the  samples  generated  for  the  study. 
After  we  were  reasonably  assured  of  having  no  multiple  minima, 
numerical  sear  h  for  the  minimum  sum  of  squares  of  residuals  was 
carried  out  as  follows: 

(a)  S(p)  is  initially  evaluated  at  p  =  Pg^'  d>  Pg^.  P()  '>+  d, 
where  d  >  0. 

'V 

(b)  Pick  the  value  of  p  that  corresponds  to  die  smallest  S(p, 

(2) 

in  Step  (a)  above.  Call  this  value  pQ  . 

Steps  (a)  and  (b)  make  up  t lie  first  iteration. ,  In  the  second  iteration 

^  *2'  j  *2)  (2 )  d 

S(p)  is  e\  '  ated  at  0  =  0q  -  j  ,  Pq  .  Pq  +  77.  In  general,  in  t he 
itli  iteration,  the  function  is  being  evaluated  at 


0 


(i) 

•0 


,  i  - 1 


(i) 


In  our  program,  we  set 


0 


(1). 

0 


0,  d 


0.5  and  i 


2 _  10. 


A  FORTRAN  program  for  the  above  search  procedure  is  given  in 
Appendix  D. 
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Appendix  D 

COMPUTER  PROCRAM  FOR  GENERATING  A  SAMPLE  FOR  THE  MONTE  CARLO  STUDY 

4- 

A  computer  program  was  developed  for  generating  the  independent 
variables  and  dependent  variable  for  a  given  combination  of  parameter 
values,  (See  Tables  1  and  16.) 

This  appendix  describes  how  to  prepare  inputs  to  this  program. 

The  correspondence  between  the  notation  used  below  and  that  of  the 

main  text  is  as  follows: 

RHO  ■  true  value  of  p 

GAM  *  vector  of  true  values  of  Y^8 

KZV[i]  *  j  in  Eq.  (11)  for  specifying  the  ith 

independent  variables  in  structures  1,  2,  and  3. 

KZV^[i]  =  j  in  Eq.  (11)  for  specifying  the  kth  element 

of  the  ith  independent  variable  for  structures 
A  ,  5 ,  and  6 . 

INPUT 

Five  input  cards  and  three  methods  are  explained.  The  first 

three  cards  remain  the  same  and  provide  input  for  all  three  methods. 

The  use  of  card:0  A  and  5  varies,  depending  on  the  method. 

First  Card.  Contains  7  integers  each  of  field  width  3: 

Col.  1-3:  K  (usually  S  A;  program  modification  required  if  greater 
than  A) 

Col,  A-6 :  T  (*200) 

Col.  7-9:  NTIMES  -  N  *  number  of  cases  to  be  run. 

^This  was  programmed  by  R.  J.  Clasen  of  the  Computer  Sciences 
Department  at  RAND. 
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Col.  10-12:  NRN  (>0) .  This  means  that  NRN-1  runs  will  be  copied  from 
tape  NTAP2  onto  tape  NTAP  and  then  this  run  will  be  writ¬ 
ten  as  the  NRNth  run  on  tape  NTAP .  If  NRN  ~  l,  twAP2 
need  not  be  specified. 

Col.  13-15:  ITYPE  =  1  if  first  method  of  Z  matrix  input  is  used.  It 
specifies  the  Z  matrices  for  Structures  1,  2,  and  3. 

ITYPE  >  1  if  second  method  of  Z  matrix  input  is  used.  It 
specifies  the  Z  matrices  for  Structures  4,  5,  and  6. 

ITYPE  =  0  if  third  method  of  Z  matrix  input  is  used.  It 
specifies  the  Z  matrices  of  Structures  7  and  8. 

The  value  for  ITYPE  is  the  number  of  KZV  cards  to  be  read 
in  when  ITYPE  >  1. 

Col.  16-18:  NTAP — the  FORTRAN  tape  unit  number  of  the  binary  tape  on 
which  the  results  will  be  written. 

Col.  19-21:  NTAP2— the  FORTRAN  tape  number  of  the  ol  d  tape  which  is 
copied  onto  NTAP.  Usually  NTAP  -  8  and  NTAP 2  -  9,  cr 
NT.d>2  -  8  and  NTAP  -  9. 

Second  Card.  Columns  1-12  contain  the  value  of  RHO  punch  with  a 
decimal  point.  (FORMAT  (F12.6)). 

Third  Card.  Contains  the  vector  GAM  (-GAMMA),  with  12  columns 
per  number  (6F12.6). 


o 


(F12.6) . 
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Methcd  2  (ITYPE  >  1) 

The  input  for  method  2  consist*  of  the  first  three  cards  above5 

plus  ITYPE  cards  of  the  form  of  the  fourth  card  of  method  1.  Let  K 

be  the  ith  number  on  the  jth  such  card.  Then  KZV.  [i], 

Ic 

Z [1]  ■  vector  of  all  ones. 

ITYPE 

Z[i]  -  i  Yj  r[kzv.  [ill  for  i  >  1. 
k-1  J 

Note  that  each  KZV  card  contains  space  for  K  numbers,  but  that  KZV[1] 
is  never  used  (KZV[2]  is  in  columns  4-6,  etc.). 

Method  3  (ITYPE  “  0) 

The  Z  matrix  is  read  in  from  T  cards,  each  card  containing  a  row 
of  the  Z  matrix.  The  T  cards  follow  the  3  cards  that  are  common  to 
all  methods.  The  first  column  of  Z  Is  set  to  1;  hence  each  card  need 
contain  only  K-1  numbers.  FORMAT  203  in  the  MAIN  (LU)  program  is  used. 
Currently,  this  format  is  (10X,  8F10.5),  but  this  may  change  in  the 
future  for  the  convenience  of  card  punching. 

INPUT  FOR  THE  TAPE  POSITIONING  AND  PRINTING  PROGRAM 

The  input  for  this  program  consists  of  one  card  with  three  inte¬ 
gers  punched  in  fields  of  three-column  width  (313). 

Col.  1-3:  NTAP  -  the  FORTRAN  unit  on  which  the  tape  is  mounted 
Col.  4-6:  NRN 
Col.  7-9:  NOGO 

If  NOGO  -  0,  the  program  will  rewind  NTAP,  and  will  then  space  NRN-1 
runs  forward  on  the  tape,  so  that  the  tape  will  be  sitting  at  the 
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beginning  of  the  run  NRN.  If  NOGO  j  0,  the  tape  will  be  positioned 
as  above,  but  run  NRN  will  then  be  printed  in  a  compact  form.  After 
printing  is  completed,  the  tape  will  be  rewound. 

| 

i 


) 


i 

t 

j 

i 

) 


i 
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